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Several  months  ago,  Dr.  Nancy  Spruill,  director  of  Acquisition  Resources  and  Analysis,  Of¬ 
fice  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (OUSD 
(AT&L)),  solicited  support  from  the  Defense  Acquisition  University  (DAU)  to  help  uncover 
the  causal  factors  that  could  be  interfering  with  attainment  of  the  Obligation  and  Expen¬ 
diture  rate  goals  of  the  Office  of  the  Secretary  of  Defense  (OSD). 

Decades  earlier,  OSD  instituted  these  goals  as  a  benchmark  to  help  weapon  systems  program  offices  maintain 
the  required  execution  pace  of  appropriated  funding.  However,  due  to  a  number  of  internal  and  external  factors. 
Department  of  Defense  (DoD)  acquisition  programs  have  sometimes  found  it  difficult  to  meet  these  goals. 

To  learn  more  about  the  intervening  obstacles,  DAU  with  assistance  from  OSD  developed  a  comprehensive  survey 
that  queried  experienced  and  high-level  DoD  personnel  involved  in  a  weapon  program's  decision  chain.  What  we 
learned  from  the  subsequent  analysis  confirmed  several  previous  suspicions.  The  data  also  indicated  the  preva¬ 
lence  of  more  underlining  perception  variances  among  many  of  the  factors  that  could  be  undermining  program 
execution  itself. 


Tremaine  is  an  associate  dean  for  outreach  and  mission  assistance  at  the  Defense  Acquisition  University's  West  Region  with  more  than  30 
years  of  acquisition  experience  in  program  management  and  systems  engineering.  Seligman  is  a  program  management  analyst  at  DAU  West 
Region  with  more  than  20  years  of  experience  in  developing  business  applications,  performing  system  analyses,  and  conducting  research. 
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The  study  results  were  presented  to  Assistant  Secre¬ 
tary  of  Defense  for  Acquisition  Katharina  McFarland 
and  other  senior  OSD  personnel.  It  also  reinforced  the 
value  of  the  memorandum  on  the  disposition  of  DoD's 
unobligated  funds,  which  was  signed  jointly  by  Under 
Secretary  of  Defense  (AT&L)  Frank  Kendall  and  Under 
Secretary  of  Defense  (Comptroller)  Robert  F.  Hale. 

Recommendations  Up  Front 

Based  on  the  research  findings  of  this  study,  there  are  a 
number  of  impact  factors  above  x  (i.e.,  above  that  mean) 
that  if  addressed  sufficiently  could  help  lower  the  barri¬ 
ers  to  attainment  of  OSD's  obligation  and  expenditure 
rate  goals.  Specifically: 

•  Institute  an  Obligation  and  Expenditure  baseline  ad¬ 
justment  for  programs  affected  by  any  funding  delay 
or  limitation  (especially  Continuing  Resolution  Au¬ 
thority  [CRA]),  then  measure  a  program's  progress 
to  that  revised  adjustment. 


•  More  thoroughly  review  the  entire  contracting  action 
value  chain.  Look  closely  at  efficiency  opportunities 
along  the  review  and  decision  cycle  continuum,  es¬ 
pecially  from  the  time  a  request  for  proposal  (RFP) 
is  developed  to  the  time  a  contract  is  let.  Set  reason¬ 
able  time  thresholds  with  triggers  that  afford  more 
proactive  measures  by  program  managers  (PMs)  and 
confirm  productivity. 

•  Establish  a  recurring  communication  forum  among 
key  stakeholders,  especially  PMs  and  OSD,  to  dia¬ 
logue  more  frequently  and  eliminate  perception  gaps 
that  could  be  creating  counterproductive  actions  and 
misconceptions. 

•  Track  requirement  changes  throughout  a  program's 
life  and  look  more  strategically  at  the  effects  on 
program  execution  and  accompanying  Acquisi¬ 
tion  Program  Baselines  (APBs).  Despite  Acquisition 
Category  (ACAT)  Levels,  there  is  an  obvious  ripple 
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Checks  and  balances  within 
the  DoD's  acquisition 


community  are  a  vital 
constituent  component  of 
program  execution— but 
every  review  should  have 
a  distinctive  purpose,  exit 
criteria,  and  associated 
suspense  date  that  are  just 
as  material  and  credible. 


effect  associated  with  any  substantive  change  in  pro¬ 
gram  content  across  a  program's  life  that  should  be 
codified  more  comprehensively.  However,  there  also  are  is¬ 
sues  associated  with  different  ACAT  levels. 

•  Review  the  program  review  cycle  and  streamline  wherever 
possible.  Checks  and  balances  within  the  DoD's  acquisition 

Table  1.  Individual  Respondent  Groups 


community  are  a  vital  constituent  component  of  program 
execution— but  every  review  should  have  a  distinctive  pur¬ 
pose,  exit  criteria,  and  associated  suspense  date  that  are 
just  as  material  and  credible. 

•  Build  and  maintain  realistic  spend  plans,  measure  against 
them,  account  for  contingencies  and  make  adjustments  with 
required  frequency  due  to  real  world  realities.  Collaborate 
with  senior  leadership  early  enough  about  required  adjust¬ 
ments  to  avoid  more  draconian  measures  later. 

•  Validate  the  key  personnel  shortage  areas  and  recognize  the 
time  it  takes  to  rebuild  those  experience  levels. 

•  Nurture  experience  in  key  functional  areas  with  strong 
catalysts  such  as  disciplined  on-the-job-training  programs, 
mentoring,  and  guidance.  With  the  recent  surge  of  contract¬ 
ing  specialist  interns,  their  progress  as  a  group  should  be 
measured  more  carefully. 

•  Evaluate  the  real  effects  of  reprogramming  action  or  realign¬ 
ment  of  future  budget  decisions  before  any  corrective  action 
is  taken. 

•  Conduct  a  wholesale  review  of  the  program  execution  met¬ 
rics  currently  in  place  and  determine  their  usefulness  and  ef¬ 
fectiveness.  What  are  they  actually  measuring?  Consolidate 
whenever  practical  and  eliminate  those  that  have  outlived 
their  usefulness. 

Research  Methodology 

Two  hundred  and  twenty-nine  DoD  personnel  responded  to 
this  survey.  The  respondents  were  comprised  of  program  of¬ 
fice  personnel  (program  managers  (PMs),  deputy  PMs,  bud¬ 
get  and  financial  managers,  and  contracting  officers),  program 
executive  officers  (PEOs),  and  their  chief  financial  officers 
(CFOs),  and  a  variety  of  senior  staff  at  OSD,  including  Head¬ 
quarter  Financial  Management  (FM)  senior  staff  and  Senior 
Acquisition  Executive  (SAE)  staff  (Table  1).  Because  several 
functional  areas  saw  lower  response  rates,  a  more  detailed 
analysis  of  the  causal  factors  was  restricted  to  an  aggregate 
sample  size  given  the  confidence  levels  required  to  draw  any 
inferences  or  conclusions. 


Survey  Respondent  Details 

ACAT  Levels 

Respondent  Groups 

TOTALS 

Respondent 

Distribution' 

1 

II 

III 

Program 

Office^ 

PEO^ 

Senior 

StafH 

Responses 

Queried 

Response 

Rate 

Total 

91 

28 

23 

142 

63 

24 

229 

698 

33% 

^  Includes  sampling  from  all  Components  and  several  DoD  agencies 

^  Program  managers,  deputy  program  managers,  budget  and  financial  managers  (BFM),  deputy  BFMs,  and  contracting 


officers 

^  Program  executive  officers  (PEOs),  deputy  PEOs  and  their  chief  financial  officers 
Headquarters  financial  managers  and  senior  acquisition  executive  staff 
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Figure  1.  Factor  Categories 
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Respondents  ranked  the  impact 
of  64  factors  under  nine  catego¬ 
ries  (Figure  1).  The  researchers 
then  assessed  the  rankings  using 
a  top  box  three  methodology  (i.e., 
averaging  the  percentages  of  5, 6, 
and  7  on  a  Likert  scale  from  1  to 
7).  Since  the  frequency  of  occur¬ 
rence  for  some  factors  also  could 
be  contributing  to  the  interfer¬ 
ence,  the  researchers  included 
an  additional  selection  (e.g.,  daily, 
weekly,  monthly,  etc.)  to  isolate 
any  potential  ignition  areas. 

Discussion 
The  Causal  Factors 
Contributing  to  Low 
Obligation  and 
Expenditure  Rates 
Figure  2  shows  the  distribution 
of  all  64  factors  assessed.  Three 
factors  reported  an  impact  rating  of  two  standard  deviations 
(also  called  sigma  [o])  above  the  mean  (denoted  by  -i-2o);  six 
factors  reported  an  impact  rating  of  one  standard  deviation 
above  the  mean  (denoted  by  -t-lo);  and  22  factors  fell  above 
an  average  (also  called  x-bar  [£])  impact  rating  (denoted  by 
x).  The  remaining  33  factors  fell  below  x. 

Nineteen  of  the  22  factors  measured  for  frequency  of  occur¬ 
rence  resulted  in  an  impact  rating  above  39  percent.  Some¬ 
times,  just  one  occurrence  appeared  to  have  a  significant 
impact. 


Figure  2.  Factor  Ranking  Distribution 

25  r 


Table  2  accounts  for  the  31  factors  above  the  mean.  They 
were  the  only  ones  further  evaluated  in  this  study  unless  a 
factor  shifted  above  x  after  any  further  specific  delineation 
(e.g.  ACAT  levels,  military  departments,  agencies,  etc.).  The 
individual  factors  showed  widespread  perception  disparities 
(see  Low  vs.  High  columns  in  Table  2)  among  the  respondent 
groups  for  the  factors  that  fell  below  -i-2o.  After  analyzing  the 
specific  individual  factors  among  all  the  respondents,  seven 
of  the  31  factors  had  an  unusually  large  o.  As  a  result  of  these 
conspicuous  gaps,  we  turned  to  the  qualitative  data.  We  also 
watched  for  any  strong  correlations  (e.g.,  positive  quantita¬ 
tive  correlation  coefficients  (r)  >  0.7 
or  qualitative  comments)  to  better 
understand  the  reasons  for  the  differ¬ 
ences  as  well  as  the  influence  of  any 
intervening  and/or  moderating  factor 
couplings.  The  remaining  discussion 
addresses  the  31  impact  factors  in  de¬ 
scending  order  from  highest  to  lowest. 

The  Factors  that  Ranked  Two 
Standard  Deviations  above  the 
Mean  (i.e.,  +  2a) 

This  first  grouping  (Table  2,  factors 
F^-F^)  indicated  release  of  full  obliga¬ 
tion/budget  authority  due  to  Continu¬ 
ing  Resolution  Authority  (CRA)  (F^), 
contract  negotiations  delays  (F^),  and 
contract  award  delays  (F^)  all  rose 
above  2cj.  The  occurrence  of  CRA  had 
the  most  significant  negative  impact 
to  Obligation  and  Expenditure  rates.  It 
also  had  one  of  the  smallest  variances 
(a)  among  the  respondent  groups. 
Even  with  the  expectation  that  CRA 


17 


Defense  AT&L:  July-August  2013 


might  prevail  and  the  subsequent  plan¬ 
ning  that  followed  for  such  a  likely  event, 
many  PMs  pointed  to  an  overly  conserva¬ 
tive  and  slow  internal  vetting  process  that 
created  additional  obstacles  in  meeting 
OSD  goals. 

Several  PMs  recommended  using  some 
sort  of  "CRA  variable"  to  temporarily 
offset  the  consequences  of  CRA  if  the  re¬ 
quired  funds  were  not  released  as  origi¬ 
nally  projected.  Next  in  rank  order  were 
contract  negotiations  and  contract  award 
delays.  The  respondents  emphasized  that 
DoD  could  fix  the  problems  more  read¬ 
ily  since,  unlike  CRA,  they  were  under 
internal  control.  When  asked  what  could 
be  done  to  reduce  the  adverse  effects  of 
all  three  factors,  the  respondents  recom¬ 
mended  the  "inclusion  of  more  risk  mitiga¬ 
tion  into  contract  award  planning,  more 
realistic  timelines,  more  realistic  plans, 
greater  funding  stability,  reduction  in  bu¬ 
reaucratic  obstacles,  more  synchronized 
internal  processes,  and  better  aligned  ac¬ 
counting  systems." 

The  Factors  that  Ranked  One 
Standard  Deviations  above  the 
Mean  (i.e.  +la) 

The  second  line  of  demarcation  (Table 
4,  factors  F'^-F^)  contained  a  majority  of 
contracting-related  factors  (i.e.,  short¬ 
age  of  contracting  officers  (F'^),  con¬ 
tractor  proposal  prep  delays  (F^),  RFP 
prep  delays  (F®),  and  source  selection 
(F^)  predominated.  Nearly  all  the  fac¬ 
tors  showed  the  emergence  of  a  more 
alarming  a  between  the  individual  re¬ 
spondent  groups— as  high  as  18  percent 
in  one  case  (i.e.,  proposal  prep  delays 
[F®]).  For  this  particular  factor,  procur¬ 
ing  contracting  officers  (PCOs)  reported 
the  highest  impact  while  PMs  ranked  it 
as  the  lowest.  Senior  staff  cited  that 
shortage  of  contracting  officers  (COs) 
(F^)  created  the  highest  impact  while 
PCOs  reported  it  had  the  lowest  impact. 
With  a  7  percent  a,  it  was  the  lowest 
among  all  six  factors  in  this  grouping. 

Given  that  six  of  the  top  nine  factors  in 
were  contract-specific  factors  that  ranked 
above  +1o,  it  came  as  little  surprise  to  see 
so  many  reinforcing  comments  surface. 


Table  2.  Impact  Factor  Ratings  in  Aggregate 
Descending  Order  With  Respondent  Group  Low  and 
High  Ratings 


Respondent  Groups 

rectors  l~jl 

Low 

High 

X 

a 

F'  Late  release  of  full  obligation/budget 
authority  due  to  CRA 

69% 

63% 

78% 

71% 

6% 

Contract  negotiations  delays 

67% 

60% 

79% 

70% 

8% 

F^  Contract  award  delays 

67% 

60% 

79% 

68% 

8% 

+  2a~ 

67% 

F'^  Shortage  of  contracting  officers 

64% 

54% 

74% 

64% 

7% 

F^  Congressional  mark 

61% 

55% 

77% 

63% 

8% 

F^  Contractor  proposal  prep  delays 

60% 

45% 

88% 

65% 

18% 

P  OSD  directed  RMD  adjustment 

58% 

43% 

70% 

60% 

10% 

F®  RFP  prep  delays 

57% 

52% 

79% 

59% 

13% 

F’  Source  selection  delays 

55% 

38% 

74% 

58% 

12% 

+la  = 

53% 

F'“  Unrealistic/overly  optimistic  spend  plans 

52% 

34% 

86% 

58% 

19% 

F’'  Changes  in  user  requirements 

51% 

33% 

72% 

56% 

14% 

F'^  Changes  to  program  acquisition  strategy 

51% 

40% 

75% 

54% 

14% 

F’^  Changes  in  other  stakeholder 
requirements 

50% 

39% 

67% 

51% 

9% 

F“  Preparing  DAE  level  review  and  decision 

50% 

44% 

54% 

50% 

6% 

F’^  Lack  of  decision  authority  at 
expected  levels 

50% 

40% 

82% 

52% 

16% 

Implementation  of  new  OSD/ 

Service  policy 

49% 

30% 

8% 

55% 

19% 

F^^  Component  directed  POM  adjustment 

49% 

35% 

61% 

48% 

10% 

F^^  Awaiting  reprogramming  action 

49% 

32% 

82% 

51% 

19% 

F^^  Changes  in  user  priorities 

47% 

39% 

55% 

49% 

6% 

F^”  Realistic  spend  plans  but  risks  materialized 

45% 

35% 

80% 

48% 

18% 

F2i  Program  delays  resulting  from  additional 
development,  testing  or  other  prerequisite 
events 

44% 

32% 

59% 

46% 

11% 

F^^  DCAA  administrative  actions 

44% 

33% 

60% 

45% 

10% 

F^^  Unplanned  congressional  adds  to  PB 
request 

43% 

31% 

66% 

44% 

13% 

P*  Use  of  undefinitized  contract  action  delays 

42% 

17% 

56% 

43% 

15% 

P^  Expenditure  contingent  on  hardware 
delivery 

41% 

17% 

59% 

42% 

16% 

P^  Loss  of  funding  through  reprogramming  ac¬ 
tion  to  higher  priority  req'ts  to  PEO  portfolio 

41% 

33% 

55% 

43% 

7% 

F^^  Lack  of  experience  levels  in  key  acquisition 
functional  areas 

40% 

24% 

56% 

44% 

14% 

F^®  Awaiting  DAE  level  review  and  decision 

40% 

30% 

65% 

44% 

19% 

P’^  Shortage  of  Cost  Estimators 

40% 

27% 

52% 

41% 

8% 

E^”  Shortage  of  business/finance  personnel 

39% 

26% 

57% 

43% 

12% 

F3i  Programmatic  conflicts  between  govern¬ 
ment  and  prime  contractor 

39% 

22% 

67% 

44% 

17% 

X  =  39% 

*lncludes  All  Responses  (AR) 
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•  "Lack  of  experienced  and  qualified  contract  specialists  ... 

•  "Alarmingly  low  personnel  qualified  ...  many  unsure/lack 
guidance  and  experience  ... ." 

•  Significantly  stressed  with  overtime  to  complete  all  con¬ 
tracting  actions  prior  to  close  of  fiscal  year." 

•  "Inadequate  training ...  inordinate  number  of  interns  with 
very  low  experience  in  all  career  fields." 

•  "Lack  of  sufficient  legal  personnel  trained  in  Acquisition." 

•  "Loss  in  brain  trust  and  skill  to  develop  complete,  clear 
SOWs  [statements  of  work]  using  proactive  contract 
language." 

•  "SOW  writing  and  the  teaching  of  SOW-writing  classes  is 
greatly  left  to  contractors  or  support  contractors,  result¬ 
ing  in  unclear  language." 

The  highest  frequency  of  occurrence  also  was  associated 
with  contracting-related  factors  (Figure  3).  By  far.  Shortage 
of  Contracting  Officers  (F^)  was  reported  as  the  single  high¬ 
est  frequency  among  all  22  factors  measured  for  frequency. 
Because  the  contracting  activity  timeline  generally  has  lengthy 
durations,  any  disruption  appears  to  have  an  unmistakable  im¬ 
pact  on  contract  award.  F^  was  seen  has  having  the  most  sig¬ 
nificant.  As  an  aggregate  group,  the  respondents  said  multiple 
contracting  actions  were  having  compounding  consequences. 


The  two  remaining  factors  above  -Oo,  Congressional  marks 
(F^)  and  OSD  directed  RMD  adjustment  (F^),  had  a  very  low 
frequency  of  occurrence  but  still  reported  a  very  high  impact 
similar  to  CRA.  When  combining  all  three,  they  appear  to  be  a 
strong  antecedent  force  (or  moderating  factor)  to  the  already 
time-consuming  chain  of  contracting  actions. 

The  Factors  that  Ranked  Above  the  Mean  (i.e.  x) 

This  final  grouping  (Table  2,  factors  F’°-F^0  accounted  for 
the  remaining  22  impact  factors.  Perception  polarities  per¬ 
sisted  especially  between  two  respondent  groups— senior 
staff  outside  the  program  office  and  PMs  inside  program  of¬ 
fices.  As  a  result  of  the  PMs'  selections  in  every  case  except 
one  (i.e..  Component-directed  Program  Objective  Memoran¬ 
dum  (POM)  adjustment  [F^^]),  the  impact  factors  ranked  well 
below  X.  In  sharp  contrast,  senior  staff  in  every  case  except 
one  (i.e..  Component-directed  POM  adjustment  [F^^])  stated 
the  majority  of  top  31  factors  had  the  largest  impact. 

Even  though  the  remaining  impact  factors  above  x  still  are 
significant,  the  researchers  shifted  the  focus  to  the  presence  of 
any  strong  correlations  since  factor  couplings  could  be  having 
a  moderating  effect  and  require  a  closer  look. 

The  Factors  that  Correlate 

Table  3  summarizes  the  strongest  and  weakest  factor  cor¬ 
relations  for  all  respondents  queried.  Several  strong  cor- 


Figure  3.  Scatter  Plot  of  Impact  Factors  with  Frequency 


Scatter  Plot  Results  >  x  (for  impact) 
5.0 


4.5 


4.0 


■  m 

"  II  3.5 

S  1!^ 

>^ 

U 

c 

(D 

13 

f  3.0 


2.5 


2.0 

3.5  4.0  4.5  5.0  5.5 


Impact  a  =  .73 
x=3.82 


1  "Late  release  of  full  obligation/budget  authority  due  to  CRA 

2  **Contract  negotiation  delays 

3  ^‘Contract  award  delays 

4  ^‘Shortage  of  contracting  officers 

5  "Congressional  mark 

6  "Contractor  proposal  prep  delays 

7  **OSD  directed  RMD  adjustment 

8  **RFP  prep  delays 

9  **Source  selection  delays 
70  *Unrealistic/overly  optimistic  spend  plans’ 

11  ‘Changes  in  user  requirements 
72  *Changes  to  program  acquisition  strategy’ 

13  ‘Changes  in  other  stakeholder  requirements 

74  *Preparing  DAE  level  review  and  decision’ 

75  *Lack  of  decision  authority  at  expected  levels’ 

16  ‘Implementation  of  new  OSD/Service  policy 

17  ‘Component  directed  POM  adjustment 

18  ‘Awaiting  reprogramming  action 

19  ‘Changes  in  user  priorities 

20  *Realistic  spend  plans  but  risks  materialized’ 

21  ‘Program  delays  from  additional  development,  testing  or  other  prere¬ 
quisite  events 

22  *DCAA  administrative  actions’ 

23  ‘Unplanned  Congressional  adds  to  PB  request 

24  ‘Use  of  undefinitized  contract  action  delays 

25  *Expenditure  contingent  on  hardware  delivery’ 

26  ‘Loss  of  funding  through  reprogramming  action  to  higher  priority  require 

ments  to  PEO  portfolio 

27  *Lack  of  Experience  levels  in  key  acquisition  functional  areas’ 

28  *Awaiting  DAE  level  review  and  decision’ 

29  ‘Shortage  of  Cost  Estimators 

30  ‘Shortage  of  business/finance  personnel 

31  ‘Programmatic  conflicts  between  goverment  and  prime  contractor 


"  >  +2ct 
"  > +1ct 
*  >x 

’  Impact  without  frequency 


19 


Defense  AT&L:  July-Augusf  2013 


Table  3.  Factor  Correlation  Couplings 


r2*  Strongest  Correlation  Coefficients 


.84 

.78 

71% 

61% 

Experience  and  Training  and  Tenure: 

F^^  Key  Acquisition  Experience  Levels  &  F'^®  Inadequate  Training 

F^^  Key  Acquisition  Experience  Levels  &  F'^^  Tenure  of  PM  &  Other  Key  Positions 

.81 

76% 

Administrative  Actions: 

F“  DCMA  &  P^  DCAA 

.82 

.70 

67% 

49% 

Changes  In  Program  Content: 

F'’  User  Requirements  &  F'®  User  Priorities 

F'^  Stakeholder  requirements  &  F'^  User  Priorities 

.71 

.70 

50% 

49% 

Contract-related  Activities 

F^  Contractor  Proposal  Delay  &  P  Contract  Negotiations  Delays 

F^  Contract  Award  Delays  &  P  Contract  Negotiations  Delays 

*  The  higher  the  %  the  stronger  the  direction  and  strength  of  the  linear  relationship  between 
the  variables 

Factors  #  1-3  > +2ct;  Factors  #  4-9  > +1ct  Factors  #  10-31  >  x 

Weakest  Correlation 


F'  Late  release  of  full  obligation/budget  authority 
due  to  CRA 

Shortage  of  Contracting  Officers 
F^  Congressional  mark/Rescission 
P  OSD-Directed  RMD  Adjustment 
F®  RFP  prep  delays 

F'°  Unrealistic/overly  optimistic  spend  plans 
F'^  Changes  to  program  acquisition  strategy 
F'^  Lack  of  decision  authority 
F'^  Implementation  of  new  OSD/Service  policy 
F'^  Component  Directed  POM  Adjustment 
F'®  Awaiting  reprogramming  action 
F^°  Realistic  spend  plans  but  risks  materialized 
F^'  Program  delays  from  prerequisite  events 
F^^  Unplanned  Congressional  adds  to  PB  request 
F^^  Expenditure  contingent  on  hardware  delivery 
Funding  Loss:  reprogramming  action  to  higher 
priority  requirements  to  PEO  portfolio 
F^®  Shortage  of  Cost  Estimators 
F^“  Shortage  of  business/finance  personnel 
F3i  Programmatic  conflicts  between  government 
and  prime  contractor 


relations  surfaced  for  factors  above  x.  User  Requirements 
(F^^)  and  User  Priorities  (F^®)  were  correlated  very  strongly. 
In  three  specific  instances,  two  factors  above  x  were 
correlated  very  strongly  with  three  factors  that  fell  below 
x:  key  acquisition  experience  (F^^)  and  inadequate  train¬ 
ing  (F"^®);  key  acquisition  experience  (F^^)  and  tenure  of 
PM  and  other  key  positions  (F'^®);  and  Defense  Contract 
Management  Agency  (DCMA)  administration  actions  (F®®) 
and  Defense  Contract  Audit  Agency  (DCAA)  administra¬ 
tion  (F®®).  Three  contract-related  factors  (F^,  F®,  and  F®) 
showed  weaker  correlations  than  expected.  To  learn  more, 
we  performed  a  regression  test  and  found  that  shortage  of 
contracting  officers  (F^)  fell  below  x  for  Air  Force  respon¬ 
dents  only.  Specific  Acquisition  Categories  (ACATs)  also 
behaved  as  a  moderating  variable.  RFP  prep  delays  (F®)  fell 
below  for  ACAT  Ns  only;  and  source  selection  (F®)  fell  below 
X  for  ACAT  Is  and  ACAT  lls  only.  A  factor  having  a  weak 
correlation  doesn't  mean  it  had  any  less  importance,  but 
any  course  of  action  intended  to  mitigate  the  presence  of 
any  impact  factor  strongly  correlated  with  another  should 
be  weighed  more  heavily  in  any  recommended  action.  For 
example,  the  turnover  of  PMs  could  be  part  of  the  experi¬ 
ence  quotient. 

Factor  Plotting  by  Impact  and  Frequency 

The  researchers  generated  a  scatter  plot  diagram  (Figure  3) 
that  punctuated  how  the  31  factors  fluctuated  between  impact 
and  frequency  of  occurrence.  In  some  cases,  the  impact  of 
certain  factors  had  low  frequencies  of  occurrence.  In  other 
cases,  the  frequency  could  be  compounding  the  impacts. 

Respondent  Comments  Regarding  the  Factors 

The  respondents  also  were  asked  several  open-ended  ques¬ 
tions  about  the  use  of  metrics  they  found  that  helped  them 


better  meet  OSD  goals  as  well  as  any  process  improvements 
they  would  recommend.  They  said  the  metrics  making  a 
difference  for  them  included  "real-time  monitoring,  frequent 
reviews,  tight  coupling  to  contractor  actions  and  milestones, 
and  realistic  spend  plans  with  inch  stones."  As  far  as  neces¬ 
sary  improvements  to  current  processes,  the  respondents 
recommended  including  a  CRA  duration  variable  that  re¬ 
adjusted  expectations,  establishing  more  realistic  program 
goals,  ensuring  more  funding  stability,  reducing  bureau¬ 
cratic  obstacles  and  streamlining  more  outdated  processes, 
increasing  cooperation  between  government  and  industry, 
and  synchronizing  disparate  accounting  systems  used  in 
obligation/expenditure  reporting. 

The  respondents  provided  a  number  of  qualitative  comments 
that  reinforced  the  quantitative  data,  especially  for  the  factors 
above  >  x  that  were  causing  obligation  rate  interference: 

Personnel,  Tools  and  Training 

•  "Takes  too  long  to  get  Acquisition  Strategies  and  Acquisi¬ 
tion  Plans  written  and  approved." 

•  "Personnel  do  not  have  experience  with  the  subject  mat¬ 
ter." 

Contracting  Activities 

•  "Inadequate  proposals,  protracted  negotiations,  lengthy 
audits,  and  lengthy  pre-award  processes." 

Requirements  Stability 

•  "Had  to  defer/reprioritize  requirements  execution  into 
FY13  and  carry  forward  FY12  funding  into  FY13  to  cover 
cutbacks/shortfall." 

•  "Changes  in  requirements  precipitated  by  other  stake¬ 
holders'  actions." 
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•  "Ill-defined  requirements." 

•  "User  leadership  routinely  changes  requirement  and 
priorities." 

Business  Ops 

•  "MIPR  billing  process  can  delay  expenditures  from  90  to 
120  days." 

•  "Delays  in  negotiating  best  deal  for  government  and 
sometimes  delays  in  getting  acceptable  proposals." 

Senior  Level/Executive  Reviews 

•  "Extensive  reviews,  too  long  to  get  decision  briefs  through 
oversight  layers— not  always  value  added." 

•  "Multiple  instances  where  milestone  documentation  took 
upward  of  9  months  to  a  year  to  get  approved." 

Funding  Realities 

•  "The  problem  isn't  unrealistic  or  overly  optimistic  spend 
plans  as  much  as  it's  not  knowing  when  funds  will  be 
appropriated  and  how  much  will  be  apportioned  by  the 
executing  organization." 

Summary 

On  Feb.  5,  2013,  we  shared  the  results  of  this  study  with  As¬ 
sistant  Secretary  McFarland  and  other  key  OSD  senior  staff. 


With  the  metrics  that  Mrs.  McFarland  has  planned  to  institute 
with  Better  Buying  Power  (BBP)  2.0,  DoD  will  have  another 
means  to  address  many  of  the  impact  factors  associated  with 
this  study  and  a  host  of  other  variables  encumbering  program 
execution  expectations. 

On  Sept.  10, 2012,  Under  Secretary  (AT&L)  Kendall  and  Under 
Secretary  (Comptroller)  Hale  jointly  signed  a  memorandum 
that  listed  sixtenets  that  could  help  combat  some  of  the  same 
factors  discussed  in  this  study  regarding  the  disposition  of 
DoD's  unobligated  funds.  Over  time,  realization  of  these  tenets 
might  also  reduce  perception  disparity  gaps  among  the  key 
personnel  who  have  a  hand  in  ensuring  our  warfighters  con¬ 
tinue  to  get  the  weapon  systems  they  need— and  on  time— to 
best  support  our  national  military  strategy. 

Authors'  Note:  The  authors  extend  our  deepest  gratitude  to  three 
key  people  involved  in  this  study.  John  Higbee  provided  exceptional 
support  as  a  thinking  partner.  Lt.  Col.  Rob  Pittman  served  as  our 
OSD  point  man  and  provided  extraordinary  support  throughout  this 
study  from  survey  inception  to  final  presentation.  Shandy  Arwood 
also  played  a  vital  role.  Her  survey  development  and  analysis  skills 
played  a  large  part  in  the  success  of  this  study.  ^ 


The  authors  can  be  contacted  at  robert.tremaine(a)dau.mil  and  donna. 
seligman(S>dau.niil. 


Program  Managers 

«-TOOL  KIT 

https://pmtoolkit.dau.mil/ 


The  Program  Managers  e-Tool  Kit  provides 
the  program  management  resources 
of  the  popular  print  Program 
Managers  Tool  Kit  in  a  dynamic 
Web-based  format. 

The  e-Tool  Kit  features: 

■  Continual  content  updates 

■  Live  policy  links  . 

■  Links  to  informative  ACQuipedia  articles 
and  related  communities  of  practice. 


Visit  https://pmtoolkit.dau.mil/ ^ 
today  to  explore  this  convenient  tooUl 
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